A nine-lesson path through how Databricks stores, skips, compacts, and maintains Delta data. Built from the official Azure Databricks docs (verified June 2026). Finish able to design a table's layout, defend it in an interview, and debug it in production.
A write makes files → layout decides what queries can skip → data skipping is the payoff → compaction & auto file sizing keep files right-sized → liquid clustering is the modern layout → predictive optimization runs the maintenance for you.
Go in order — each lesson builds on the last. After each one, answer its question without looking. Roughly 5 hours total, plus notebook runtime.
| # | Lesson | Time | Self-check |
|---|---|---|---|
| 01 | Traditional writes & small files | 25m | Why does a Spark write make many small files, and why is that slow to read? |
| 02 | Partitioning (and when not to) | 35m | When should you partition, and why is partitioning on customer_id bad? |
| 03 | Data skipping & Z-ordering | 40m | What stats does Delta keep per file, and how does ZORDER BY help? |
| 04 | OPTIMIZE / compaction | 30m | What does OPTIMIZE do, why is it idempotent, and when do you need OPTIMIZE FULL? |
| 05 | Optimized writes | 30m | How do optimized writes work, and which operations enable them automatically? |
| 06 | Auto compaction | 30m | How is auto compaction different from optimized writes and predictive optimization? |
| 07 | Auto optimize & autotuning | 35m | What's the autotuned target file size for a 5 TB table, and what doesn't auto optimize replace? |
| 08 | Liquid clustering | 45m | Why can you change clustering keys with no rewrite, and why is it incompatible with partition/ZORDER? |
| 09 | Deletion vectors (merge-on-read) | 35m | How do deletion vectors make DELETE/UPDATE/MERGE cheap, and how are the soft-deletes applied physically? |
| 10 | VACUUM, time travel & retention | 40m | Why does VACUUM limit time travel, and which two retention dials must you raise to time-travel further back? |
| 11 | Predictive optimization | 35m | What does predictive optimization run, on which tables, and how do you enable it? |
OPTIMIZE), cleanup (VACUUM), and stats (ANALYZE).
OPTIMIZE, or an explicit delta.targetFileSize. Lessons 02–07 teach exactly when.
true): 128 MBOPTIMIZE FULL: DBR 16.0+ · convert partitioned: 18.1+delta.deletedFileRetentionDuration)